Ecological Validity Research Articles

From the lab to the classroom: Improving children's prospective memory in a natural setting.

Laboratory-based studies have shown that children's ability to remember intentions (i.e., prospective memory; PM) can be improved by asking them to imagine performing the PM task beforehand (i.e., episodic future thinking; EFT) or to predict their PM performance. Moreover, combining the two strategies resulted in an additional improvement in children's PM performance. However, the effectiveness of these encoding strategies on real-life PM tasks is still unknown. The aim of the present study was to evaluate the effect of EFT instructions, performance predictions, and of their combination on children's PM in a natural setting, namely in the classroom. Twelve classes composed by a total of 121 children (53% females) aged between seven and 9 years participated to the study. As a PM task, children were asked by their teachers to deliver a letter to their parents and to bring it back to school the next day. Children were divided into four groups: control, prediction, EFT, and the EFT + prediction group. Parent reports on children's everyday prospective and retrospective memory failures were also collected. Results showed that encoding strategies were effective in enhancing children's PM performance. However, differences compared to previous laboratory-based findings emerged since predicting PM performance resulted to be most effective in enhancing real-life PM performance. Moreover, parent reports were related to children's PM performance. These novel findings highlight the importance of studying PM interventions in natural settings in order to increase their ecological validity and inform educational practices.

Tracking and predicting the treatment adherence of patients under rehabilitation: a three-wave longitudinal validation study for the Rehabilitation Adherence Inventory

This study aimed to develop and validate a new measurement tool, the Rehabilitation Adherence Inventory (RAI), to measure patients’ rehabilitation adherence. We recruited 236 patients with anterior cruciate ligament (ACL) ruptures from the United Kingdom (Mage = 33.58 ± 10.03, range = 18 to 59; female = 46.2%). Participants completed a survey, that measured their rehabilitation adherence, rehabilitation volume, psychological needs support, autonomous motivation, and intention at baseline, and at the 2nd and 4th month. Factorial, convergent, discriminant, concurrent, predictive, ecological validity and test–retest reliability of the RAI were tested via exploratory factor analysis (EFA), confirmatory factor analysis (CFA), and structural equation modelling (SEM). All the EFAs, CFAs, and SEMs yielded acceptable to excellent goodness-of-fit, χ2 = 10.51 to 224.12, df = 9 to 161, CFI > 0.95, TLI > 0.95, RMSEA <0.09 [90%C I < 0.06 to 0.12], SRMR <0.04. Results fully supported the RAI’s factorial, convergent, discriminant, and ecological validity, and test–retest reliability. The concurrent and predictive validity of the RAI was only partially supported because the RAI scores at baseline was positively associated with rehabilitation frequency at all time points (r = 0.34 to 0.38, p < 0.001), but its corresponding associations with rehabilitation duration were not statistically significant (p = 0.07 to 0.93). Overall, our findings suggest that this six-item RAI is a reliable and valid tool for evaluating patients’ rehabilitation adherence.

Associations Between Hypoglycemia Awareness Status and Symptoms of Hypoglycemia Among Adults with Type 1 or Insulin-Treated Type 2 Diabetes Using the Hypo-METRICS Smartphone Application.

Introduction: This study examined associations between hypoglycemia awareness status and hypoglycemia symptoms reported in real-time using the novel Hypoglycaemia-MEasurement, ThResholds and ImpaCtS (Hypo-METRICS) smartphone application (app) among adults with insulin-treated type 1 (T1D) or type 2 diabetes (T2D). Methods: Adults who experienced at least one hypoglycemic episode in the previous 3 months were recruited to the Hypo-METRICS study. They prospectively reported hypoglycemia episodes using the app for 10 weeks. Any of eight hypoglycemia symptoms were considered present if intensity was rated between "A little bit" to "Very much" and absent if rated "Not at all." Associations between hypoglycemia awareness (as defined by Gold score) and hypoglycemia symptoms were modeled using mixed-effects binary logistic regression, adjusting for glucose monitoring method and diabetes duration. Results: Of 531 participants (48% T1D, 52% T2D), 45% were women, 91% white, and 59% used Flash or continuous glucose monitoring. Impaired awareness of hypoglycemia (IAH) was associated with lower odds of reporting autonomic symptoms than normal awareness of hypoglycemia (NAH) (T1D odds ratio [OR] 0.43 [95% confidence interval {CI} 0.25-0.73], P = 0.002); T2D OR 0.51 [95% CI 0.26-0.99], P = 0.048), with no differences in neuroglycopenic symptoms. In T1D, relative to NAH, IAH was associated with higher odds of reporting autonomic symptoms at a glucose concentration <54 than >70 mg/dL (OR 2.18 [95% CI 1.21-3.94], P = 0.010). Conclusion: The Hypo-METRICS app is sensitive to differences in hypoglycemia symptoms according to hypoglycemia awareness in both diabetes types. Given its high ecological validity and low recall bias, the app may be a useful tool in research and clinical settings. The clinical trial registration number is NCT04304963.

Ecological validity of neuropsychological interventions: A systematic review.

The concept of ecological validity (EV) in neuropsychological interventions (NI) has been consistently advocated, but there is a lack of reviews focused on how EV is operationalized in NI programmes. This review aims to address this gap by exploring the outcome measures more commonly used for assessing EV and to understand the main characteristics of programmes with good EV. A literature search was conducted to identify studies that examined the EV of NI programmes, following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and the Cochrane Collaboration Guidelines. A total of twenty-seven studies were included in this review. Among these, only three studies explicitly described the procedures used to assess EV. Additionally, almost half of the studies assumed that interventions had good EV based on the characteristics of the programmes. The inconsistent assessment of EV of NI programmes prevented the identification of specific characteristics of programmes demonstrating good EV. This systematic review reveals a significant gap in the literature concerning the operationalization of EV within the field of NI. Further research is required to establish a consistent definition of EV in the context of NI and to develop criteria for its effective operationalization.

Sentence processing in bilingual children

Abstract Research in sentence processing in bilingual children is emergent but incomplete as very few studies examine the processing of structurally complex sentences or bilingual children’s real-time interpretation of sentences. One underexplored linguistic feature which can offer insights in this direction are garden-path sentences, i.e., sentences with temporary syntactic ambiguity. These are difficult to process for monolingual children as incremental processing results in an initial misinterpretation and the need for reanalysis. Studies on bilingual children’s processing of garden-path sentences have used paradigms with limited ecological validity and which are not informative about one’s interpretation while listening. This study bridges this gap by investigating the processing of garden-path sentences in bilingual children with the visual-world eye-tracking paradigm. It further explores the role of referential context in the visual stimuli to aid disambiguation. Monolingual and bilingual children aged 8–11 years completed a task similar to Trueswell et al. (1999). The results showed similar difficulty with revising garden-path sentences as evidenced by comprehension accuracy for both groups but only the monolinguals showed real-time garden-path effects in the gaze data. We interpret these findings as a manifestation of slower sentence processing in bilingual children. Both groups made limited use of the referential context to facilitate processing.

Ecological validity of common behavioral decision making tasks: evidence across two samples

ABSTRACT Background Clinicians and scholars routinely use behavioral decision tasks to assess real-world decision making capabilities. However, many common behavioral decision making tasks lack data on the extent to which they predict real-world risky behaviors. Across two pre-registered studies, and two timepoints, we assessed decision making abilities using common behavioral tasks and predicted participants’ real-world risky decision making from task performance. Method In Study 1, 918 Mechanical Turk (MTurk) workers completed three decision making tasks in addition to assessments of real-world risk behavior: preventive health behaviors, COVID-19 vaccination status, and virtual social distancing task performance. In Study 2, 221 college student participants completed the Study 1 tasks plus additional assessments of decision making and real world risk and protective behaviors. Results Across both studies, the selected behavioral decision tasks rarely predicted real world behavior and, when they did, the relationship was weak at best. Conclusions These data suggest that these behavioral decision making tasks may not be good predictors of real world risky behavior at present, with some evidence that the specificity of the behavior being assessed matters (i.e. the closer the task was to the specific behavior being predicted), calling for additional ecological validity research, with a greater variety of tasks in the future.

Exploring experiential differences in everyday activities – A focused ethnographic study in the homes of people living with memory-led Alzheimer's disease and posterior cortical atrophy

BackgroundSupporting ageing in place, quality of life and activity engagement are public health priorities for people living with dementia, but little is known about the needs and experiences of community-dwelling people with rarer forms of dementia with lesser known symptoms. Posterior cortical atrophy (PCA) is a rare form of dementia usually caused by Alzheimer's disease but which is characterised by diminished visual processing (rather than a dominant memory problem), which poses challenges for maintaining independence and accessing appropriate support. MethodsThis study used a comparative qualitative design and focussed ethnographic methods to explore experiential differences in activity engagement for 10 people with the most common, memory-led presentation of Alzheimer's disease and 10 people with posterior cortical atrophy within their everyday home environments. ResultsWhile the data collection revealed much rich variation in individual and contextual factors, some tentative high-level differences in the experiences of everyday activities could be drawn out, seemingly attributable to the different diagnoses' differing dominant symptoms. These included people with posterior cortical atrophy being less likely to use environmental cues to initiate activities, and more likely to withhold from asking for support because of preserved insight into the impact of this on carers. This lack of initiation of activities could be misinterpreted as apathy. People with posterior cortical atrophy also were discouraged from engaging in activities by disorientation within the home, and difficulties localising, identifying and manipulating objects. People with the more common, memory-led presentation of Alzheimer's disease exhibited more memory-based difficulties with engaging with activities such as forgetting planned activities, where to locate the items required for an activity and the steps involved. Despite these distinct symptom-led challenges, all participants and their family members demonstrated resourcefulness and resilience in making creative adaptations to support continued engagement in everyday activities, supporting the widely reported management strategies of people with dementia of the Alzheimer's type more generally. ConclusionsThese findings offer helpful insights into some the differing impacts dementia related visual and memory impairments can have on everyday activity engagement, which will be helpful for others navigating these challenges and the health and social care practitioners working with people affected by these conditions. The findings also highlight the vast individual variation in the multitude of individual and contextual factors involved in everyday activity engagement, and suggest important areas for future work utilising methods which are similarly high in ecological validity and accessibility as the home-based focussed ethnographic methods utilised here.

Investigating the impact of greenery elements in office environments on cognitive performance, visual attention and distraction: An eye-tracking pilot-study in virtual reality

The human-nature connection is one of the main aspects determining supportive and comfortable office environments. In this context, the application of eye-tracking-equipped Virtual Reality (VR) devices to support an evaluation on the effect of greenery elements indoors on individuals’ efficiency and engagement is limited. A new approach to investigate visual attention, distraction, cognitive load and performance in this field is carried out via a pilot-study comparing three virtual office layouts (Indoor Green, Outdoor Green and Non-Biophilic). 63 participants completed cognitive tasks and surveys while measuring gaze behaviour. Sense of presence, immersivity and cybersickness results supported the ecological validity of VR. Visual attention was positively influenced by the proximity of users to the greenery element, while visual distraction from tasks was negatively influenced by the dimension of the greenery. In the presence of greenery elements, lower cognitive loads and more efficient information searching, resulting in improved performance, were also highlighted.

Virtual reality games for cognitive rehabilitation of older adults: a review of adaptive games, domains and techniques

In recent decades, the senior adults population worldwide has increased, as well as the medical conditions related to aging, such as cognitive decline. Virtual reality (VR) games are a valuable addition to conventional cognitive rehabilitation as they increase engagement to the therapy through customization, socialization, immersion, and feedback. This review, performed according to PRISMA protocol, addresses the following questions: How VR games have been used for cognitive rehabilitation?, What cognitive domains have been addressed by VR games and in which populations have these games been used?, Which features have been considered to improve engagement in VR games for cognitive rehabilitation?, How is the difficulty adjustment of exercises carried out in VR games for cognitive rehabilitation?. We found 25 scientific works related to these questions, 92% of them treating one cognitive domain at a time, despite the fact that the related literature recognizes the value of training multiple domains simultaneously. Our review indicates that, despite the existence of serious VR games for working memory training, such as those described in Flak et al. (Front Psychol 10:807, 2019. https://doi.org/10.3389/fpsyg.2019.00807), to our knowledge, there are no applications that simultaneously address multiple cognitive domains and incorporate dynamic difficulty adjustment, which are important to ensure ecological validity of therapy and therapy adherence, respectively. In addition, we found that games themselves could be used to monitor the user’s progression. It is also important to determine the impact of multiplayer interactions in the game, test difficulty adjustment approaches that use physiological variables, and define difficulty-skill relationships aligned with the user’s preferences. This paper concludes that the main barriers to implement dynamic difficulty adjustment in VR games for cognitive rehabilitation are: (i) the absence of metrics to estimate when the game offers to the players a challenge adapted their skills, and (ii) the lack of a conceptual framework that integrates relevant theories such as state of flow, cognitive load, cognitive rehabilitation, and feedback systems.

A Serious Game for Cognitive Stimulation of Older People With Mild Cognitive Impairment: Design and Pilot Usability Study.

Cognitive stimulation of older people helps prevent, and even treat, age-related diseases, such as mild cognitive impairment. Playing games reduces the probability of experiencing this pathology, which is related to the loss of the ability to carry out some instrumental activities of daily living. This work describes the design and development of a serious game for the cognitive stimulation of older people, with exercises related to the daily life task of shopping. A pilot study for its preliminary usability validation is also presented. The designed serious game includes 4 exercises consisting of shopping in a hypermarket, ordering products, making payments, and organizing the purchase, thus dealing with the most frequent cognitive problems of older people associated with episodic declarative memory, naming, calculation, and organization, respectively. A total of 19 older people participated in the pilot study for the usability validation of the serious game. They indicated that they like the aesthetic and interesting topic of the game. They reported that it provides a high level of entertainment and could be useful in daily life for mental stimulation. The participants found the serious game to be intuitive, but the ease of use and readability of the instructions could be improved. This study suggests that the innovative serious game developed could be accepted by older people for their cognitive stimulation to prevent or treat mild cognitive impairment, although a long-term intervention study should be performed as future work. Its ecological validity design, with everyday tasks, adaptable levels of difficulty, and motivational mechanisms, is a differentiating factor compared to similar serious games.

Retrieving autobiographical memories in autobiographical contexts: are age-related differences in narrated episodic specificity present outside of the laboratory?

The Autobiographical Interview, a method for evaluating detailed memory of real-world events, reliably detects differences in episodic specificity at retrieval between young and older adults in the laboratory. Whether this age-associated reduction in episodic specificity for autobiographical event retrieval is present outside of the laboratory remains poorly understood. We used a videoconference format to administer the Autobiographical Interview to cognitively unimpaired older adults (N = 49, M = 69.5, SD = 5.94) and young adults (N = 54, M = 22.5, SD = 4.19) who were in their homes at the time of retrieval. Relative to young adults, older adults showed reduced episodic specificity in their home environment, as reflected by fewer episodic or "internal" details (t (101) = 3.23, p = 0.009) and more "external" details (i.e., semantic, language-based details) (t (101) = 3.60, p = 0.003). These findings, along with detail subtype profiles in the narratives, bolster the ecological validity of the Autobiographical Interview and add promise to the use of virtual cognitive testing to improve the accessibility, participant diversity, scalability, and ecological validity of memory research.

Integrating historical patterns and future trends for ecological management zone identification and validation: A case study in Beijing, China

Ecological management zones (EMZs) are pivotal in improving the management of ecosystem services (ESs) and promoting sustainable regional development. In this study, we developed a comprehensive framework aimed at identifying EMZs and substantiating their efficacy through the amalgamation of historical evolutionary patterns and future trends. We applied this framework to Beijing, China, and selected five vital ESs for the study area namely, water yield (WY), carbon sequestration (CS), habitat quality (HQ), soil conservation (SC) and water purification (WP). The framework involves two key components. Firstly, the identification of EMZs is based on the historical evolution of five types of ESs and the dynamic assessment of ES bundles. Subsequently, it enables a simulation of various scenarios to predict future alterations in land use and ESs, thereby validating the effectiveness of the identified EMZs. Our findings reveal notable spatial heterogeneity among different ESs, and that CS, HQ, SC, and WP exhibited synergies, while WY and showed trade-offs with the remaining four types of ESs. Based on an analysis of ES bundle evolution trajectories, we identified four types of EMZs: ecological conservation zone, ecological restoration zone, ecological transition zone and sustainable construction zone. Through strategic EMZ planning, it becomes possible to augment the area of forestland and grassland, alleviate the contradiction between arable land and construction land, and enhance the supply of various ESs. The proposed framework not only offers a novel perspective on the scientific management of ESs but also furnishes decision-makers and planners with an intuitive understanding of the tangible benefits associated with EMZ planning.

Scaling success for low-income, first-generation in college, and/or racially minoritized students through a culture of ecological validation.

Scaling success for low-income, first-generation in college, and/or racially minoritized students through a culture of ecological validation.

Acoustic characteristics of voice production in virtual reality-simulated and physical environments: a comparative study in university professors

This study investigated the reliability of a virtual reality-simulated classroom to generate a comparable self-perception of voice quality and acoustic effects of phonation to a real classroom in a group of teachers, and sense of presence. Thirty university professors participated in the study and were required to produce loud connected speech by reading a 100-word text in two conditions: (1) in a real classroom including a group of students, and (2) in a virtual replica of the classroom consisting of a 360-degree video of the same classroom and students, which was displayed using a head mounted display. Ambient noise was controlled in both conditions by playing classroom noise through headphones. The self-perception of voice quality, the long-term average spectrum and smooth cepstral peak prominence were estimated in both conditions. The sense of presence generated by virtual reality was measured after interacting with the virtual classroom. There were no statistically significant differences in the self-perception of voice quality or in the acoustic measures of voice production between conditions. The sense of presence in the virtual classroom was high. Our findings suggest that a virtual reality-simulated classroom generate comparable self-perception of voice quality and acoustic effects of phonation to the real classroom, and a high sense of presence, in a group of teachers. Additionally, it is important to highlight the potential of virtual reality to enhance the ecological validity of acoustic assessment of voice production in laboratories and clinical settings.

Preliminary Evidence for Global Properties in Human Listeners During Natural Auditory Scene Perception.

Theories of auditory and visual scene analysis suggest the perception of scenes relies on the identification and segregation of objects within it, resembling a detail-oriented processing style. However, a more global process may occur while analyzing scenes, which has been evidenced in the visual domain. It is our understanding that a similar line of research has not been explored in the auditory domain; therefore, we evaluated the contributions of high-level global and low-level acoustic information to auditory scene perception. An additional aim was to increase the field's ecological validity by using and making available a new collection of high-quality auditory scenes. Participants rated scenes on 8 global properties (e.g., open vs. enclosed) and an acoustic analysis evaluated which low-level features predicted the ratings. We submitted the acoustic measures and average ratings of the global properties to separate exploratory factor analyses (EFAs). The EFA of the acoustic measures revealed a seven-factor structure explaining 57% of the variance in the data, while the EFA of the global property measures revealed a two-factor structure explaining 64% of the variance in the data. Regression analyses revealed each global property was predicted by at least one acoustic variable (R2 = 0.33-0.87). These findings were extended using deep neural network models where we examined correlations between human ratings of global properties and deep embeddings of two computational models: an object-based model and a scene-based model. The results support that participants' ratings are more strongly explained by a global analysis of the scene setting, though the relationship between scene perception and auditory perception is multifaceted, with differing correlation patterns evident between the two models. Taken together, our results provide evidence for the ability to perceive auditory scenes from a global perspective. Some of the acoustic measures predicted ratings of global scene perception, suggesting representations of auditory objects may be transformed through many stages of processing in the ventral auditory stream, similar to what has been proposed in the ventral visual stream. These findings and the open availability of our scene collection will make future studies on perception, attention, and memory for natural auditory scenes possible.

Communication Under Sharply Degraded Auditory Input and the "2-Sentence" Problem.

Despite performing well in standard clinical assessments of speech perception, many cochlear implant (CI) users report experiencing significant difficulties when listening in real-world environments. We hypothesize that this disconnect may be related, in part, to the limited ecological validity of tests that are currently used clinically and in research laboratories. The challenges that arise from degraded auditory information provided by a CI, combined with the listener's finite cognitive resources, may lead to difficulties when processing speech material that is more demanding than the single words or single sentences that are used in clinical tests. Here, we investigate whether speech identification performance and processing effort (indexed by pupil dilation measures) are affected when CI users or normal-hearing control subjects are asked to repeat two sentences presented sequentially instead of just one sentence. Response accuracy was minimally affected in normal-hearing listeners, but CI users showed a wide range of outcomes, from no change to decrements of up to 45 percentage points. The amount of decrement was not predictable from the CI users' performance in standard clinical tests. Pupillometry measures tracked closely with task difficulty in both the CI group and the normal-hearing group, even though the latter had speech perception scores near ceiling levels for all conditions. Speech identification performance is significantly degraded in many (but not all) CI users in response to input that is only slightly more challenging than standard clinical tests; specifically, when two sentences are presented sequentially before requesting a response, instead of presenting just a single sentence at a time. This potential "2-sentence problem" represents one of the simplest possible scenarios that go beyond presentation of the single words or sentences used in most clinical tests of speech perception, and it raises the possibility that even good performers in single-sentence tests may be seriously impaired by other ecologically relevant manipulations. The present findings also raise the possibility that a clinical version of a 2-sentence test may provide actionable information for counseling and rehabilitating CI users, and for people who interact with them closely.

Digital Neuropsychology beyond Computerized Cognitive Assessment: Applications of Novel Digital Technologies.

Compared with other health disciplines, there is a stagnation in technological innovation in the field of clinical neuropsychology. Traditional paper-and-pencil tests have a number of shortcomings, such as low-frequency data collection and limitations in ecological validity. While computerized cognitive assessment may help overcome some of these issues, current computerized paradigms do not address the majority of these limitations. In this paper, we review recent literature on the applications of novel digital health approaches, including ecological momentary assessment, smartphone-based assessment and sensors, wearable devices, passive driving sensors, smart homes, voice biomarkers, and electronic health record mining, in neurological populations. We describe how each digital tool may be applied to neurologic care and overcome limitations of traditional neuropsychological assessment. Ethical considerations, limitations of current research, as well as our proposed future of neuropsychological practice are also discussed.

STEM in the classroom: a scoping review of emerging research on the integration of STEM education within Australian schools

AbstractThe paper presents a scoping review of research that has been conducted on integrated STEM programmes in Australian schools over a 10-year period (2012–2022). It aims to determine how integrated STEM is being practiced. A total of 17 papers were chosen for review. The review explored the major characteristics of research that has been conducted on integrated STEM programmes in Australian schools, the teaching strategies used to teach integrated STEM and the reported impacts of these programmes. In fifteen of the papers experts from outside the school such as scientists and engineers, usually academics, were involved. Cooperative learning, project/problem-based learning and authentic experiences were the most common teaching strategies in a field characterised by positive learner outcomes. It is evident that teaching strategies are significant in the integration of STEM. Future research should focus on addressing issues of ecological validity, sustainability and scalability to ensure as many students as possible reap the benefits of high-quality STEM education.

Design of a neuro rehabilitation protocol with ecological validity for the postural control of children with Down syndrome

Design of a neuro rehabilitation protocol with ecological validity for the postural control of children with Down syndrome

The Morality Game: Is the Evaluation of the Instrumental Utility of Moral Dilemmas Related to Decision Outcomes

This article investigates the relation of the evaluation of the instrumental utility of moral dilemmas to decision outcomes. The results indicate that nuances of dilemma perception assessed in the study (perceived realism of dilemmas, their plausibility, reflection of real behavior and an assessment of the extent to which responses to moral dilemmas can be used to judge moral maturity, empathy, level of intelligence and sense of humor) are not significantly related to objective behavior, as expressed in the number of consequentialist responses. Instruction about the nature of the study, however, may influence the moral choices made. Participants who were informed by instruction that their personality could be judged by their responses to moral dilemmas gave more deontological responses than others. An explanation is proposed that such instruction acts as a factor that provokes the adoption of the role of the apprehensive participant and, as a consequence, socially desirable behavior. This aspect highlights the importance of considering contextual factors in the study of moral behavior. It is suggested that moral dilemmas should not be viewed through the lens of ecological validity but rather as a tool that allows for a high level of experimental control, facilitating a more indepth study of behavior under specific stimulus conditions